Skip to content

feat: CI/CD pipeline generation (generate ci / cd / ci-cd)#330

Open
mitanuriel wants to merge 149 commits intodevelopfrom
feature/ci-pipeline-generation
Open

feat: CI/CD pipeline generation (generate ci / cd / ci-cd)#330
mitanuriel wants to merge 149 commits intodevelopfrom
feature/ci-pipeline-generation

Conversation

@mitanuriel
Copy link
Copy Markdown

Delivers end-to-end CI/CD pipeline generation via three new sync-ctl generate subcommands.

What's included:

generate ci — project-aware CI pipeline (Clippy, tests, Docker Buildx, Trivy SARIF, Gitleaks)
generate cd — platform-specific CD workflows for Azure (app-service, aks, container-apps), GCP (cloud-run, gke), and Hetzner (vps, hetzner-k8s, coolify)
generate ci-cd — single command that generates both in one pass
Unresolved secrets emitted as {{TOKEN}} placeholders with a companion SECRETS_REQUIRED.md
--dry-run flag on all three commands for terminal preview without writing files

feat(agent): add extended thinking, conversation compaction, and UI i…
bug(wrong ref for rig-bedrocks) wrong referenced rig-bedrock package
fix: add version to rig-bedrock dependency for crates.io publishing
mitanuriel and others added 28 commits March 30, 2026 17:14
- CloudBuildConfig: steps + optional artifacts.objects + timeout
- Every step is a Docker container: name is the image URI (node:20, etc.)
- runtime_docker_image() maps Actions identifiers to Docker Hub images
- shell_step() uses bash -c for all arbitrary commands (install/lint/test/build)
- Docker steps use gcr.io/cloud-builders/docker natively
- Image scan: aquasec/trivy image; secret scan: zricethezav/gitleaks image
- Artifact upload: top-level artifacts.objects GCS path (no step)
- Cache step intentionally skipped: no GCS bucket info at generation time
- No trigger block: GCP triggers are console/API only
- 23 unit tests; 1391 total passing
- CiConfig: all fields Option<T> + #[serde(default)] so partial configs valid
- load_ci_config(): tries .syncable.ci.toml first, then [ci] in .syncable.toml
- merge_config_into_context(): applies config layer before CLI flags
- parse_platform/parse_format: normalise strings to CiPlatform/CiFormat enums
- CiContext: new fields config_test_command, env_prefix, skip_steps, extra_branches
- test_helpers: make_minimal_context() alias for CI-22 tests
- 15 unit tests; 1406 total passing
- CD-02: CdContext struct + collect_cd_context entry point
  - Enums: CdPlatform, DeployTarget, Registry, MigrationTool, Environment
  - Detection: Terraform, K8s manifests, Helm charts, migration tools, health checks
  - 20 unit tests

- CD-17: CdPipeline schema (platform-agnostic IR)
  - Step structs: AuthStep, RegistryStep, DockerBuildPushStep, MigrationStep,
    TerraformStep, DeployStep, HealthCheckStep, RollbackInfo, NotificationStep,
    EnvironmentConfig
  - UnresolvedToken with {{PLACEHOLDER}} formatting
  - 17 unit tests

- Token resolver: two-pass resolution engine for CD tokens
  - Deterministic pass resolves PROJECT_NAME, IMAGE_NAME, REGISTRY_URL, etc.
  - Placeholder pass collects unresolved tokens with deduplication
  - 12 unit tests

- CD-22: cd-manifest.toml writer
  - Writes [resolved], [unresolved], and [environments] sections
  - Supports optional fields with skip_serializing_if
  - 7 unit tests

All 58 tests pass. Module wired via cd_generation/mod.rs.
…D-06)

- CD-03: registry.rs — ACR, GAR, GHCR login steps + image tag strategy
  + render_registry_login_yaml() for each registry type
  + build_image_tag() / build_gar_image_tag() helpers
  + registry_secrets_doc_entries() for SECRETS_REQUIRED.md
  + 30 unit tests

- CD-04: auth_azure.rs — Azure OIDC auth step (azure/login@v2)
  + generate_azure_auth() config + render_azure_auth_yaml()
  + OIDC permissions block helper
  + 20 unit tests

- CD-05: auth_gcp.rs — GCP WIF auth step (google-github-actions/auth@v2)
  + generate_gcp_auth() config + render_gcp_auth_yaml()
  + GAR Docker auth helper (gcloud auth configure-docker)
  + 24 unit tests

- CD-06: auth_hetzner.rs — Hetzner SSH + kubeconfig auth
  + VPS pattern: webfactory/ssh-agent + known_hosts
  + K8s pattern: kubeconfig write from secret
  + 28 unit tests

All 160 cd_generation tests pass. Zero clippy warnings.
…CD-11)

- CD-07 deploy_azure.rs: App Service, AKS, Container Apps deploy generators
- CD-08 deploy_gcp.rs: Cloud Run, GKE deploy generators
- CD-09 deploy_hetzner.rs: VPS (SSH), HetznerK8s, Coolify deploy generators
- CD-10 migration.rs: 7 migration tools (Flyway, Liquibase, Alembic, Django, Prisma, sqlx, Diesel), SSH variant for VPS
- CD-11 health_check.rs: curl with retry for HTTP targets, kubectl rollout status for K8s targets
- Wire all 5 modules into cd_generation/mod.rs

Tests: 290 passed (130 new), 0 failed. Clippy clean.
CD-18: Azure workflow template (templates/azure.rs)
  - Full deploy-azure.yml builder from CdPipeline schema
  - Renders header, OIDC auth, Docker buildx, deploy steps
    (AppService/AKS/ContainerApps), health check, rollback
  - 20 tests

CD-19: GCP workflow template (templates/gcp.rs)
  - Full deploy-gcp.yml builder from CdPipeline schema
  - Renders WIF auth, GAR Docker config, Cloud Run / GKE deploy
  - Cloud Run uses step output URL for health check
  - 18 tests

CD-20: Hetzner workflow template (templates/hetzner.rs)
  - Full deploy-hetzner.yml builder from CdPipeline schema
  - GHCR login, SSH agent (VPS/Coolify), kubeconfig (K8s)
  - VPS SSH deploy, Coolify webhook, kubectl for HetznerK8s
  - 22 tests

CD Writer (writer.rs):
  - CdFile/CdFileKind types, write_cd_files(), print_cd_dry_run()
  - Conflict detection with force flag, WriteSummary
  - 5 tests

Pipeline Builder (pipeline.rs):
  - build_cd_pipeline() assembles CdPipeline from CdContext
  - Calls auth/registry/deploy/migration/health_check generators
  - Maps context Environment to schema EnvironmentConfig
  - Default branch filters, namespaces, and replica counts
  - 25 tests

CD-01: CLI Entrypoint
  - GenerateCommand::Cd variant with --platform, --target,
    --registry, --image-name, --dry-run, --output, --force
  - CdPlatform, CdTarget, CdRegistry ValueEnum types in cli.rs
  - handle_generate_cd() in generate.rs: context → pipeline →
    resolve tokens → render template → dry-run or write
  - Wired into main.rs, lib.rs, handlers/mod.rs

381 CD tests passing (91 new), 0 failures.
- CD-12 environments.rs: Multi-environment job strategy (13 tests)
- CD-13 rollback.rs: Platform-specific rollback scripts (14 tests)
- CD-14 reusable_workflow.rs: _deploy-base.yml with workflow_call (14 tests)
- CD-15 versioning.rs: Image tag versioning strategy (12 tests)
- CD-16 terraform_step.rs: Terraform init/plan/apply steps (15 tests)
- CD-21 notification.rs: Slack deployment notifications (18 tests)
- CD-29 dispatch.rs: Manual workflow_dispatch inputs (16 tests)
- Wire terraform + notification into pipeline builder
- Register all 7 new modules in mod.rs
- 483 tests passing, 0 clippy warnings
…24/25/26/27/28)

CD-23: Combined CI+CD command (generate ci-cd)
- Added CiCd variant to GenerateCommand with platform, ci_format, target,
  registry, image_name, dry_run, output, force, notify flags
- handle_generate_cicd orchestrates both generators, cross-links secrets doc
- Wired in cli.rs, lib.rs, main.rs with telemetry tracking

CD-24: .syncable.cd.toml project-level config
- CdConfig struct with all-optional fields (platform, target, environments,
  registry, image_name, health_check_path, migration_command, default_branch)
- load_cd_config: dedicated .syncable.cd.toml takes precedence over shared
  .syncable.toml [cd] table
- merge_config_into_cd_context: config file layer between detection and CLI
- 19 unit tests

CD-27: CD secrets inventory documentation
- collect_cd_secret_names: scans rendered YAML for secrets.* references
- secret_metadata: known descriptions for Azure/GCP/Hetzner/Slack secrets
- render_cd_secrets_table: markdown table output
- generate_cd_secrets_doc: full document generator

CD-28: Hetzner prerequisites checklist
- Firewall rules (22/80/443/6443), SSH key, Docker, DNS checklist
- Auto-appended to secrets doc when platform is Hetzner

CD-25: Comprehensive unit tests (33 tests)
- cd_snapshot_tests: 24 tests covering Azure/GCP/Hetzner pipeline rendering,
  environment structure, health check, migration, terraform, notification,
  rollback, token resolution, multi-platform consistency, no hardcoded secrets
- cd_cross_linking_tests: 9 tests covering dispatch/environment consistency,
  versioning+notification composability, terraform+rollback, reusable workflow

CD-26: Integration tests (31 tests)
- Full pipeline rendering for all 8 platform/target combinations
- No hardcoded secrets validation
- Secrets doc generation for Azure/GCP/Hetzner
- Context collection from 5 language fixtures (node/python/rust/go/java)
- Config loading + merging integration
- Cross-platform consistency, health check presence, secrets expression syntax

Total: 2,164 tests passing (0 failures)
…view (CD-30)

- Added 2c. sync-ctl generate cd: full options table, platform/target
  matrix, registry defaults, CD steps, multi-env structure, config file
  (.syncable.cd.toml) docs, output files, and examples
- Added 2d. sync-ctl generate ci-cd: options table, all 4 output files,
  examples for all 3 platforms
- Updated table of contents with links to both new sections
- Completes CD-30 (Documentation & Help Text for CD Generation)
- Add `migration_command_override: Option<String>` to `CdContext`
- `merge_config_into_cd_context()` now sets the field when the config
  key is present — previously the value was parsed but silently dropped
- `build_cd_pipeline()` overrides `MigrationStep.command` with the
  user-supplied value after tool-derived default is constructed, so the
  override only applies when a migration tool is also detected
- Update all manual `CdContext` fixtures with `migration_command_override: None`
- Add tests: merge_migration_command, merge_migration_command_absent_leaves_none,
  migration_command_override_replaces_tool_default,
  migration_command_override_without_tool_produces_no_step
- README: add `generate cd` and `generate ci-cd` quick-start examples
Rust's line-continuation escape ("\) strips leading whitespace from the
first line of a format string. This caused 4 workflow steps per pipeline
to be emitted at column 0, producing invalid YAML.

Fixed by removing the backslash continuation from all affected format!()
openers in azure.rs (10 sites), gcp.rs (12 sites), hetzner.rs (13 sites).

Also gitignores generated pipeline output files (.github/workflows/deploy-*.yml,
.syncable/cd-manifest.toml, .syncable/SECRETS_REQUIRED.md) so they are
never accidentally committed.
@blocksorg
Copy link
Copy Markdown

blocksorg Bot commented Apr 24, 2026

Mention Blocks like a regular teammate with your question or request:

@blocks review this pull request
@blocks make the following changes ...
@blocks create an issue from what was mentioned in the following comment ...
@blocks explain the following code ...
@blocks are there any security or performance concerns?

Run @blocks /help for more information.

Workspace settings | Disable this message

@mitanuriel mitanuriel requested a review from Alex793x April 24, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants